Information Gain of Black-box Testing1
نویسندگان
چکیده
For model-based black-box testing, test cases are often selected from the syntactic appearance of the specification of the system under test, according to a pre-given test data adequacy criterion. We introduce a novel approach that is semantics-based, independent of the syntactic appearance of the system specification. Basically, we model the system under test as a random variable, whose sample space consists of all possible behavior sets (with respect to the specification) over the known interface of the black-box. The entropy of the system is measured as the (Shannon) entropy of the random variable. In our criterion, the coverage of a test set is measured as the expected amount of entropy decrease (i.e., the expected amount of information gained) once the test set is run. Since our criterion is syntactic independent, we study the notion of information-optimal software testing where, within a given constraint, a test set is selected to gain the most information.
منابع مشابه
Knowledge Extraction from the Neural ‘Black Box’ in Ecological Monitoring
Phytoplankton biomass within the Saginaw Bay ecosystem (Lake Huron, Michigan, USA) was characterized as a function of select physical/chemical indicators. The complexity and variability of ecological systems typically make it difficult to model the influences of anthropogenic stressors and/or natural disturbances. Here, Artificial Neural Networks (ANNs) were developed to model chlorophyll a con...
متن کاملEnsuring monotonic gain characteristics in estimated models by fuzzy model structures
We consider the situation where a non-linear physical system is identi ed from input-output data. In case no speci c physical structural knowledge about the system is available, parameterized grey-box models cannot be used. Identi cation in black box type of model structures is then the only alternative, and general approaches like neural nets, neuro-fuzzy models, etc., have to be applied. Howe...
متن کاملEnsuring Certain Physical Properties in Black BoxModels
We consider the situation where a nonlinear physical system is identiied from input-output data. In case no speciic physical structural knowledge about the system is available, parameterized grey box models cannot be used. Identiication in black-box-type of model structures is then the only alternative, and general approaches like neural networks, wavelet models, neuro-fuzzy models, etc., have ...
متن کاملAuditing Black-Box Models Using Transparent Model Distillation With Side Information
Black-box risk scoring models permeate our lives, yet are typically proprietary or opaque. We propose a transparent model distillation approach to audit such models. Model distillation was first introduced to transfer knowledge from a large, complex teacher model to a faster, simpler student model without significant loss in prediction accuracy. To this we add a third criterion transparency. To...
متن کاملAnalysing and characterising optimization problems using length scale
Analysis of optimization problem landscapes is fundamental in the understanding and characterisation of problems and the subsequent practical performance of algorithms. In this paper, a general framework is developed for characterising black-box optimization problems based on length scale,whichmeasures the change in objective function with respect to the distance between candidate solution pair...
متن کامل